Thanks for the PR, could you explain the impact that the change will have?
Could you visualise the difference between the changes for the same seed?

To visualize the difference, I changed line categoryBits=0x0001, to set categoryBits as 0x0002, allowing the bipedal walker's lower leg to overlap with the terrain rather than being resolved by the physics engine. Below is the first frame. It is evident that the original code's initial position causes an overlap between the two rigid bodies (lower leg and terrain).

Before	After

|

Ok that makes sense, @LiuShuai26 could you do some testing to investigate the impact on training performance?
Do agent get better / worse / equal scores? Do old agents using the old initialisation get better / worse / equal scores?

I ran three experiments for both the old and new environments with different seed. I am using sample-factory without tuning. It seems to yield identical results. A slight disruption in the first few frames appears to have insufficient impact on the training process.

Thanks for running the tests, what is the training algorithm used and I'm surprised that this required 40 million observation to train

Thanks for running the tests, what is the training algorithm used and I'm surprised that this required 40 million observation to train

I am using the Sample Factory with the APPO algorithm, without tuning, and the training parameters are set to default. There will be a data efficiency problem when training in a small environment because Sample Factory is designed for high throughput.

command line parameter from Sample Factory example:
--algo=APPO --use_rnn=False --num_envs_per_worker=20 --policy_workers_per_policy=2 --recurrence=1 --with_vtrace=False --batch_size=512 --reward_scale=0.1

Could you rerun this with stable baselines 3 or cleanrl?

I tried using stable-baselines3 and cleanrl initially, but both of them were not ready to go. Following the documentation to launch training resulted in multiple errors. Unfortunately, I don't have the time to address these issues.